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1  Summary 

This  project  laid  the  foundation  for  a  novel  methodology  for  correcting  er¬ 
roneous  program  executions  using  specifications  at  run-time.  The  basis  of 
the  methodology  is  a  view  of  the  specification  as  a  non-deterministic  im¬ 
plementation,  which  may  permit  a  high  degree  of  non-determinism.  The 
key  insight  is  to  use  likely  correct  actions  by  an  otherwise  erroneous  execu¬ 
tion  to  prune  the  non-determinism  in  the  specification,  thereby  transmuting 
the  specification  to  an  implementation  at  run-time  and  reducing  the  perfor¬ 
mance  overhead.  A  suite  of  techniques  and  tools  were  designed,  developed, 
optimized  and  rigorously  evaluated  in  this  project.  It  leveraged  the  Alloy 
specification  language  and  its  SAT-based  tool-set  as  an  enabling  technology 
for  specification-based  analysis.  The  ideas,  techniques,  tools,  and  evaluation 
results  from  this  project  contributed  in  part  to  44  archival  publications,  4 
completed  Masters  theses,  and  3  completed  PhD  dissertations.  This  project 
funded  in  part  8  graduate  students,  including  3  female  students. 

2  Annual  summaries 

2.1  Reporting  period:  05/01/2009  —  04/30/2010 

During  the  first  year  of  the  project,  the  following  research  contributions  were 
made: 
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•  Contract-based  data  structure  repair  -  Introduced  the  idea  of  using 
rich  behavioral  contract  specifications  including  invariants,  pre-  and 
post-conditions  as  the  basis  of  systematic  data  structure  repair. 

•  Repair  algorithms  Developed  four  algorithms  that  embody  the  idea. 
The  algorithms  leverage  MIT’s  Alloy  tool-set  to  provide  systematic 
repair,  and  employ  heuristics  to  optimize  performance. 

•  Similarity  metric  -  Used  a  distance  metric  for  graph  similarity  to  com¬ 
pute  the  effect  of  repair  on  an  erroneous  program  state  and  to  evaluate 
different  algorithms  for  effectiveness. 

•  Evaluation  -  Conducted  an  experimental  evaluation  of  the  feasibility 
of  contract-based  repair  and  demonstrated  the  promise  it  holds. 

A  basic  technique  embodying  these  ideas  and  and  experimental  evalua¬ 
tion  were  presented  at  the  24th  European  Conference  on  Object-Oriented 
Programming  (ECOOP)  in  June  2010;  a  pre-print  version  of  the  paper  is 
submitted  along  with  this  report. 

2.2  Reporting  period:  05/01/2010  -  04/30/2011 

During  the  second  year  of  the  project,  our  primary  research  contribution 
was  on  program  repair  using  data  structure  repair.  A  key  element  of  the 
“Usability”  thrust  of  our  project  is  to  design  a  repair  feedback  mechanism  to 
help  users  debug  their  code  or  specifications.  We  developed  a  novel  mecha¬ 
nism  for  translating  repair  actions  performed  on  an  erroneous  program  state 
into  code  that  abstracts  those  actions  using  assignment  statements  that  may 
replace  existing  program  statements  or  be  added  as  new  statements.  These 
statements  serve  as  debugging  suggestions,  which  the  user  can  choose  to  ap¬ 
ply  or  ignore.  Details  of  this  approach  and  an  experimental  evaluation  were 
presented  at  the  IEEE  4th  International  Conference  on  Software  Testing, 
Verification  and  Validation  (ICST)  in  March  2011;  a  pre-print  version  of  the 
paper  is  submitted  along  with  this  report. 

2.3  Reporting  period:  05/01/2011  -  04/30/2012 

During  the  third  year  of  the  project  (May  1,  2011  to  April  30,  2012),  our 
primary  research  contribution  was  to  develop  a  new  technique  to  enhance 
our  core  approach  for  data  structure  repair  to  scale  better.  Our  insight 
into  scalability  is  two-fold:  (1)  the  dynamic  program  trace  of  field  writes 
and  reads  provides  useful  guidance  to  repair  incorrect  state  mutations  by  a 
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faulty  program;  and  (2)  unsatisfiable  cores  generated  by  SAT  can  capture 
the  history  of  previous  runs,  which  can  be  used  in  an  efficient  iterative 
approach  on  successive  problems  with  increasing  state  spaces.  Details  of 
this  technique  and  an  experimental  evaluation  were  presented  at  the  18th 
International  Conference  on  Tools  and  Algorithms  for  the  Construction  and 
Analysis  of  Systems  (TACAS)  in  March  2012;  a  pre-print  version  of  the 
paper  is  submitted  along  with  this  report. 

Additionally,  we  utilized  unsatisfiable  cores  in  another  novel  technique, 
which  was  for  fault  localization  -  the  problem  of  locating  faults  in  the  source- 
code  of  a  buggy  program.  Specifically,  we  developed  a  specification-based 
technique  that  utilized  correct  and  erroneous  executions  of  the  buggy  pro¬ 
gram  to  more  accurately  locate  faults.  Our  insight  is  that  unsatisfiabil¬ 
ity  analysis  of  violated  specifications,  enabled  by  SAT  technology,  can  help 

(1)  compute  unsatisfiable  cores  that  contain  likely  faulty  statements;  and 

(2)  generate  tests  that  help  spectra-based  localization.  Details  of  this  tech¬ 
nique  and  an  experimental  evaluation  were  presented  at  the  27th  IEEE/ ACM 
International  Conference  on  Automated  Software  Engineering  (ASE)  in  Septem¬ 
ber  2012;  a  pre-print  version  of  the  paper  is  submitted  along  with  this  report. 

2.4  Reporting  period:  05/01/2012  -  06/30/2012 

During  the  final  two  months  of  the  project  (May  1,  2012  to  June  30,  2012), 
we  focused  on  enhancing  the  ideas,  analyses,  and  implementations  we  de¬ 
veloped  in  this  project  to  integrate  them  as  parts  of  doctoral  dissertations 
-  we  expect  three  future  doctoral  dissertations  to  use  the  work  done  in  this 
project  at  their  foundation. 

3  Archival  publications 

3.1  Published  after  the  end  of  the  funding  period 

1.  R.  N.  Zaeem  and  S.  Khurshid.  Test  Input  Generation  Using  Dynamic 
Programming.  In  Proc.  ACM  SIGSOFT  20th  International  Sympo¬ 
sium  on  the  Foundations  of  Software  Engineering  (FSE),  11  pages, 
Research  Triangle  Park,  NC,  Nov.  2012 

2.  L.  Zhang,  M.  Kim,  and  S.  Khurshid.  FaultTracer:  A  Change  Impact 
and  Regression  Fault  Analysis  Tool  for  Evolving  Java  Programs.  In 
Proc.  ACM  SIGSOFT  20th  International  Symposium  on  the  Founda¬ 
tions  of  Software  Engineering  (FSE),  4  pages,  Research  Triangle  Park, 
NC,  Nov.  2012.  Research  tool  demonstration  paper 


3 


3.  C.  H.  P.  Kim,  S.  Khurshid,  D.  Batory.  Shared  Execution  for  Efficiently 
Testing  Product  Lines.  In  Proc.  IEEE  International  Symposium  on 
Software  Reliability  Engineering  (ISSRE),  Dallas,  TX,  Nov.  2012 

4.  S.  Ganov,  S.  Khurshid,  and  D.  E.  Perry.  Annotation-aided  Auto¬ 
mated  Incremental  Analysis  for  Alloy  via  Domain  Specific  Solvers.  In 
Proc.  14th  International  Conference  on  Formal  Engineering  Methods 
( ICFEM ),  pages  414-429,  Kyoto,  Japan,  November  2012 

5.  S.  Roychowdhury  and  S.  Khurshid:  Localization  of  faults  in  software 
programs  using  Bernoulli  divergences.  In  Proc.  International  Sym¬ 
posium  on  Information  Theory  and  its  Applications  (ISITA),  pages 
586-590,  Honolulu,  HI,  Oct.  2012 

6.  J.  H.  Siddiqui  and  S.  Khurshid.  Scaling  symbolic  execution  using 
ranged  analysis.  In  Proc.  ACM  International  Conference  on  Object 
Oriented  Programming  Systems  Languages  and  Applications  (OOP- 
SLA),  pages  523-536,  Tuscon,  AZ,  Oct.  2012 

7.  S.  Roychowdhury.  Ensemble  of  feature  selectors  for  software  fault 
localization.  In  Proc.  IEEE  International  Conference  on  Systems, 
Man,  and  Cybernetics  (SMC),  pages  1351-1356,  Seoul,  Korea,  Oct. 
2012 

8.  S.  Roychowdhury  and  S.  Khurshid.  A  family  of  generalized  entropies 
and  its  application  to  software  fault  localization.  In  Proc.  IEEE  Con¬ 
ference  on  Intelligent  Systems  (IS),  pages  368-373,  Sofia,  Bulgaria, 
Sep.  2012 

9.  D.  Gopinath,  R.  N.  Zaeern,  and  S.  Khurshid.  Improving  the  Effec¬ 
tiveness  of  Spectra-based  Fault  Localization  using  Specifications.  In 
Proc.  27th  IEEE/ ACM  International  Conference  on  Automated  Soft¬ 
ware  Engineering  (ASE),  pages  40-49,  Essen,  Germany,  Sep.  2012 

10.  G.  Yang,  S.  Khurshid,  and  M.  Kim.  Specification-based  test  repair 
using  a  lightweight  formal  method.  In  Proc.  18th  International  Sym¬ 
posium  on  Formal  Methods  (EM),  pages  455-470,  Paris,  France,  Aug. 
2012 

11.  L.  Zhang,  D.  Marinov,  L.  Zhang,  and  S.  Khurshid.  Regression  muta¬ 
tion  testing.  In  Proc.  International  Symposium  on  Software  Testing 
and  Analysis  (ISSTA),  pages  331-341,  Minneapolis,  MN,  July  2012 
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12.  G.  Yang,  C.  Pasareanu,  and  S.  Khurshid.  Memoized  symbolic  exe¬ 
cution.  In  Proc.  International  Symposium  on  Software  Testing  and 
Analysis  (ISSTA),  pages  144-154,  Minneapolis,  MN,  July  2012 

3.2  Published  during  the  funding  period 

13.  M.  Z.  Malik  and  S.  Khurshid.  Dynamic  shape  analysis  using  spectral 
graph  properties.  In  Proc.  IEEE  Fifth  International  Conference  on 
Software  Testing,  Verification  and  Validation  (ICST),  pages  211-220, 
Montreal,  Canada,  Apr.  2012 

14.  J.  H.  Siddiqui,  D.  Marinov,  and  S.  Khurshid.  Lightweight  data-flow 
analysis  for  execution-driven  constraint  solving.  In  Proc.  IEEE  Fifth 
International  Conference  on  Software  Testing,  Verification  and  Vali¬ 
dation  (ICST),  pages  91-100,  Montreal,  Canada,  Apr.  2012 
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aware  data  structure  repair  using  SAT.  In  Proc.  18th  International 
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16.  J.  H.  Siddiqui  and  S.  Khurshid.  Staged  symbolic  execution.  In  Proc. 
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17.  L.  Zhang,  D.  Marinov,  L.  Zhang,  and  S.  Khurshid.  An  empirical 
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Learning  Technologies  in  Software  Engineering  (MALETS),  pages  11- 
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